Introductory Papers and Guides

Original Paper on DCA

2006

Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006 Nov-Dec;26(6):565-74. PMID: 17099194.

General Introduction to DCA

2016

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. PMID: 26810254.

How to Understand a Decision Curve

2019

Vickers, A.J., van Calster, B. & Steyerberg, E.W. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res 3, 18 (2019). https://doi.org/10.1186/s41512-019-0064-7

Guide for Investigators on Reporting Decision Curves

2018

Ben Van Calster, Laure Wynants, Jan F.M. Verbeek, Jan Y. Verbakel, Evangelia Christodoulou, Andrew J. Vickers, Monique J. Roobol, Ewout W. Steyerberg. Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators. European Urology, Volume 74, Issue 6, 2018, Pages 796-804, ISSN 0302-2838, https://doi.org/10.1016/j.eururo.2018.08.038.

Extensions to DCA

Applications to Survival Time Data among others

2008

Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak. 2008;8:53. Published 2008 Nov 26. doi:10.1186/1472-6947-8-53.

DCA & Treatment Response: Model Evaluation

2007

Vickers AJ, Kattan MW, Daniel S. Method for evaluating prediction models that apply the results of randomized trials to individual patients. Trials. 2007;8:14. Published 2007 Jun 5. doi:10.1186/1745-6215-8-14.

DCA & Treatment Response: Trial Design

2006

Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: a systematic approach based on risk group. Trials. 2006 Oct 5;7:30. PMID: 17022818; PMCID: PMC1609186.

Discussion Papers and Theoretical Background

General Theoretical Background to DCA (1)

2008

Vickers AJ. Decision analysis for the evaluation of diagnostic tests, prediction models and molecular markers. Am Stat. 2008;62(4):314-320. doi:10.1198/000313008X370302

General Theoretical Background to DCA (2)

2010

Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol. 2010;37(1):31-38. doi:10.1053/j.seminoncol.2009.12.004

Mathematical Underpinnings of DCA through Relationship to Relative Utility (Similar Method)

2009

Baker SG, Cook NR, Vickers A, Kramer BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc Ser A Stat Soc. 2009 Oct 1;172(4):729-748. PMID: 20069131; PMCID: PMC2804257.

Widely Cited First Prediction Model Evaluation Paper to Introduce 3-Step Approach of Discrimination, Calibration, Clinical Utility

2010

Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010 Jan;21(1):128-38. PMID: 20010215; PMCID: PMC3575184.

Editorials and Commentaries Recommending DCA in Major Journals

Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use

2016

Kerr KF, Brown MD, Zhu K, Janes H. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol. 2016 Jul 20;34(21):2534-40. PMID: 27247223; PMCID: PMC4962736.

Net Benefit Approaches to the Evaluation of Prediction Models, Molecular Markers, and Diagnostic Tests

2016

Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016 Jan 25;352:i6. PMID: 26810254; PMCID: PMC4724785.

Decision Curve Analysis

2015

Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. 2015 Jan 27;313(4):409-10. PMID: 25626037.

Beyond the Usual Prediction Accuracy Metrics: Reporting Results for Clinical Decision Making

2012

Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making. Ann Intern Med. 2012 Aug 21;157(4):294-5. PMID: 22910942.

Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models

2023

David J. McLernon, Daniele Giardiello, Ben Van Calster, et al; topic groups 6 and 8 of the STRATOS Initiative. Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models. Ann Intern Med.2023;176:105-114. [Epub 27 December 2022].

Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review

2022

de Hond AAH, Leeuwenberg AM, Hooft L, et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. npj Digit Med. 2022 Jan 20;5(1):2. doi: 10.1038/s41746-021-00549-7. PMID: 35348862; PMCID: PMC8393232.

Other Editorials

Decision curve analysis in the evaluation of radiology research

2022

Vickers AJ, Woo S. Decision curve analysis in the evaluation of radiology research. Eur Radiol. 2022 Sep;32(9):5787-5789. doi: 10.1007/s00330-022-08685-8. Epub 2022 Mar 29. PMID: 35348862.

Decision curve analysis to evaluate the clinical benefit of prediction models

2021

Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J. 2021 Oct;21(10):1643-1648. doi: 10.1016/j.spinee.2021.02.024. Epub 2021 Mar 3. PMID: 33676020; PMCID: PMC8413398.

Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis

2022

Mijderwijk HJ, Nieboer D. Is My Clinical Prediction Model Clinically Useful? A Primer on Decision Curve Analysis. Acta Neurochir Suppl. 2022;134:115-118. doi: 10.1007/978-3-030-85292-4_15. PMID: 34862535.

Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models

2021

Sadatsafavi M, Adibi A, Puhan M, Gershon A, Aaron SD, Sin DD. Moving beyond AUC: decision curve analysis for quantifying net benefit of risk prediction models. Eur Respir J. 2021 Nov 4;58(5):2101186. doi: 10.1183/13993003.01186-2021. PMID: 34735641.

Newly Added

A Systematic Review of the Literature Demonstrates Some Errors in the Use of Decision Curve Analysis but Generally Correct Interpretation of Findings

2019

Capogrosso P, Vickers AJ. Med Decis Making. 2019 Jul;39(5):493-498. PMID: 30819037.

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models

2019

Christodoulou E, et al. J Clin Epidemiol 2019;110:12–22.

A framework for making predictive models useful in practice

2021

Jung K, et al. J Am Med Inform Assoc 2021; 28 (6): 1149–58.

Calculating net benefit under resource constraints

2023

Karandeep Singh, Nigam H Shah, Andrew J Vickers, Assessing the net benefit of machine learning models in the presence of resource constraints, Journal of the American Medical Informatics Association, Volume 30, Issue 4, April 2023, Pages 668–673, https://doi.org/10.1093/jamia/ocad006

Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models

2023

McLernon DJ, et al. Ann Intern Med. 2023 Jan;176(1):105-114. PMID: 36571841.

Seven Common Errors in Decision Curve Analysis

2023

Andrew Vickers

Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review

2023

Dhiman P, Ma J, Andaur Navarro CL, Speich B, Bullock G, Damen JAA, Hooft L, Kirtley S, Riley RD, Van Calster B, Moons KGM, Collins GS. Journal of Clinical Epidemiology. 2023; ISSN 0895-4356. https://doi.org/10.1016/j.jclinepi.2023.03.012.